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ABSTRACT 


Increased  pirate  activity  in  the  Gulf  of  Aden  (GOA)  has  gotten  the  attention  of  the 
international  community,  and  many  countries  are  engaged  in  counter-piracy  operations  to 
protect  vulnerable  shipping  and  provide  humanitarian  aid.  In  February  2009,  the 
Internationally  Recommended  Transit  Corridor  (IRTC)  was  established  in  order  to 
introduce  safer  and  more  organized  passage  for  all  merchant  vessels  transiting  the  GOA. 
This  thesis  uses  simulation  to  identify  the  key  factors  involved  in  escorting  vulnerable 
shipping  through  the  Gulf  of  Aden  (GOA).  Specifically,  a  scenario  in  which  a  group  of 
merchant  ships  travels  under  escort  of  a  warship  is  modeled  using  an  agent-based 
simulation  environment.  Using  state-of-the-art  experimental  designs,  over  300,000 
counter-piracy  escort  missions  are  simulated  and  analyzed.  The  results  indicate  that 
convoys  are  most  successful  when  they  contain  fewer  than  14  merchant  ships,  travel  at 
speeds  greater  than  18  knots,  position  the  warship  in  front  or  on  the  flank  of  the  convoy, 
and  identify  pirates  at  a  range  of  no  less  than  4  kilometers.  It  is  found  that  three  or  more 
pirate  vessels  are  especially  difficult  to  counter,  as  are  pirates  travelling  at  speeds  greater 
than  39  knots. 
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EXECUTIVE  SUMMARY 


Increased  pirate  activity  in  the  Gulf  of  Aden  (GOA)  has  gotten  the  attention  of  the 
international  community,  and  many  countries  are  engaged  in  counter-piracy  operations  to 
protect  vulnerable  shipping  and  provide  humanitarian  aid.  The  European  Union  Naval 
Force  (EUNAVFOR)  Somalia — Operation  ATALANTA,  is  the  European  Union’s  first 
naval  operation.  In  February  2009,  the  Internationally  Recommended  Transit  Corridor 
(IRTC)  was  established  in  order  to  introduce  safer  and  more  organized  passage  for  all 
merchant  vessels  transiting  the  GOA.  The  IRTC  is  a  corridor  between  Somalia  and 
Yemen  within  international  waters,  consisting  of  two  lanes,  each  of  five  nautical  miles 
(nm’s)  width,  one  eastbound  and  one  westbound,  with  a  space  of  two  nm  between  them. 
The  total  length  of  the  transit  corridor  is  480  nm,  and  a  vessel  maintaining  14  knots 
requires  34.5  hours  to  pass  through  it. 

This  thesis  uses  simulation  to  identify  the  key  factors  involved  in  escorting 
vulnerable  shipping  through  the  Gulf  of  Aden  (GOA).  Specifically,  a  scenario  in  which  a 
group  of  merchant  ships  travels  under  escort  of  a  warship  is  modeled  using  an  agent- 
based  simulation  environment  known  as  Map-Aware,  Non-uniform  Automata  (MANA). 
The  simulation  is  run  to  address  the  following  questions: 

•  How  does  the  number  of  merchant  ships  affect  the  ability  of  the  frigate  to 
defend  them? 

•  How  many  pirates  can  a  convoy  adequately  handle? 

•  What  factors  should  be  taken  under  consideration  when  we  have  an 
escorted  convoy? 

We  provide  insight  into  the  critical  factors  and  threshold  values  a  decision  maker 
should  consider  when  deciding  how  many  merchant  ships  a  warship  should  escort  and 
what  tactics  should  be  used. 

The  scenario  simulated  in  MANA,  shown  in  Figure  1,  involves  coalition  force 
operations  using  a  warship  with  a  helicopter  to  protect  a  convoy  of  merchant  ships 
transiting  westbound  in  two  columns  along  the  IRTC.  This  area  usually  includes  many 
fishing  or  other  individual  traveling  vessels,  and  the  pirates  behave  like  one  of  them  until 

they  approach  a  potential  target,  attack  it,  and  attempt  to  capture  it. 

xvii 


Using  state-of-the-art  design  of  experiments  (DOE),  over  300,000  convoy 
operations  are  simulated.  The  DOE  distinguishes  between  decision  factors  (i.e.,  those 
variables  that  we  can  control)  and  noise  factors  (those  variables  we  cannot  control),  and 
varies  the  following: 

•  Decision  factors:  warship  position,  convoy  speed,  number  of  merchant 
ships,  distance  between  merchant  ships,  and  helicopter  takeoff  delay 

•  Noise  factors:  number  of  pirates,  pirate  speed,  communication  latency 
between  merchant  ships  and  warship,  and  distance  from  convoy  at  which 
pirates  reveal  identity 

Three  primary  analysis  techniques  are  used  to  quantify  how  mean  merchant  (blue) 
casualties  are  affected  by  the  factors  above:  multiple-regression  analysis,  robust  analysis, 
and  classification  and  regression  trees.  The  analysis  finds  that  in  the  scenario  modeled: 

xviii 


•  The  three  most  influential  factors  in  the  regression  analysis  are  the  number 
of  pirates,  pirate  speed,  and  convoy  speed, 

•  The  most  successful  convoys  have  fewer  than  14  merchant  ships  and 
travel  at  greater  than  18  knots, 

•  The  warship  should  patrol  in  front  or  on  the  flank  of  the  convoy, 

•  It  is  important  to  be  able  to  identify  pirates  at  4  kilometers  from  their 
target, 

•  More  than  three  pirate  vessels  are  especially  difficult  to  counter,  as  are 
pirates  that  travel  at  more  than  39  knots. 
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I.  INTRODUCTION 


Everything  should  be  made  as  simple  as  possible,  but  not  simpler. 

Albert  Einstein 


A.  WHAT  IS  PIRACY? 

The  word  pirate  derives  from  the  Greek  “peirates”  (7i8ipaxf|s),  the  name  of  an 
ancient  adventurer  who  attacked  a  ship  (Johnson  &  Valencia,  2005).  Most  people  think  of 
pirates  as  belonging  to  a  very  long-distant  era,  from  approximately  1500  to  1800,  when 
the  Barbary  corsairs  of  North  Africa  made  the  Mediterranean  Sea  rampant  with  piracy 
(Berlatsky,  2010).  In  Southeast  Asia,  piracy  has  been  present  for  approximately  2,000 
years,  but  incidents  have  increased  dramatically  in  the  last  two  decades  (Young,  2007). 
That  augmented  pirate  activity  has  alarmed  the  international  community,  and  many 
countries  are  engaged  in  counter-piracy  operations  to  protect  vulnerable  shipping  and 
humanitarian  aid. 

In  1982,  the  United  Nations  Convention  on  the  Law  of  the  Sea  (UNCLOS) 
provided  a  framework  for  the  repression  of  piracy  under  international  law.  In  particular, 
Article  101  defines  piracy  as  follows: 

Article  101 
Definition  of  piracy 

Piracy  consists  of  any  of  the  following  acts: 

(a)  any  illegal  acts  of  violence  or  detention,  or  any  act  of  depredation,  committed 
for  private  ends  by  the  crew  or  the  passengers  of  a  private  ship  or  a  private  aircraft,  and 
directed: 

(i)  on  the  high  seas,  against  another  ship  or  aircraft,  or  against  persons  or 
property  on  board  such  ship  or  aircraft; 

(ii)  against  a  ship,  aircraft,  persons  or  property  in  a  place  outside  the 
jurisdiction  of  any  State; 
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(b)  any  act  of  voluntary  participation  in  the  operation  of  a  ship  or  of  an  aircraft 
with  knowledge  of  facts  making  it  a  pirate  ship  or  aircraft; 

(c)  any  act  of  inciting  or  of  intentionally  facilitating  an  act  described  in 
subparagraph  (a)  or  (b)  (Young,  2007). 

B.  LITERATURE  REVIEW 

The  author  found  two  related  studies  addressing  maritime  counter-piracy  using 
agent-based  simulations. 

The  first  was  in  2005,  by  Walton  and  others  (Walton,  Paulo,  McCarthy,  & 
Vaidyanathan,  2005).  Their  research  was  part  of  a  larger  study  that  examined  maritime- 
domain  protection  in  the  Straits  of  Malacca.  The  scenario  involved  the  attack  of  a  small 
boat  carrying  explosives  against  a  larger,  high-value  commercial  ship,  and  was  designed 
to  demonstrate  the  quick  reaction  of  the  maritime -protection  system.  The  scenario  was 
developed  in  the  simulation  software  MANA  (Map  Aware,  Non-uniform  Automata)  and 
had  either  patrol  crafts  conducting  a  continuous  patrol  pattern  in  the  straits  or  armed  sea 
marshals  aboard  the  commercial  ship.  The  research  examined  ways  to  prevent  the  success 
of  the  small-boat  attack  when  it  exited  from  a  fairly  narrow  portion  of  the  straits  that  is 
usually  congested. 

The  second  study  was  in  2010,  conducted  by  Decraene  and  others  (Decraene, 
Anderson,  &  Yoke  Hean  Low,  2010).  The  scenario  was  a  large  commercial  vessel 
transiting  a  wide  expanse  of  water,  when  a  pirate  agent  closes  on  it.  The  commercial  ship 
employs  evasive  maneuvers  and  a  non-lethal  weapon.  Decraene  and  Anderson’s  model 
was  developed  in  MANA  to  investigate  the  requirements  for  a  large  commercial  vessel 
using  non-lethal  deterrents  to  defend  against  hijacking. 

Both  scenarios  are  dealing  with  maritime  vessel  protection,  the  first  is  using  either 
crafts  patrolling  congested  straits  or  armed  personnel  aboard  the  vessels  under  protection, 
while  the  second  is  using  self-defense  deterrents.  This  thesis  evaluates  a  scenario  using  an 
escort  warship  and  a  helicopter.  All  three  of  these  theses  used  the  MANA  simulation 
environment. 
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C.  RESEARCH  QUESTIONS 

The  goal  of  this  thesis  is  to  simulate  counter-piracy  operations  in  the  Gulf  of  Aden 
(GOA)  and  explore  the  benefits  of  escorting  vulnerable  shipping.  This  is  done  by 
simulating  a  scenario  in  which  a  group  of  merchant  ships  travels  under  escort  of  a 
warship.  The  simulation  is  run  to  address  the  following  questions: 

•  How  does  the  number  of  merchant  ships  impact  the  ability  of  the  frigate  to 
defend  them? 

•  How  many  pirates  can  the  convoy  adequately  handle? 

•  What  factors  should  be  taken  under  consideration  when  we  have  an 
escorted  convoy? 

D.  BENEFITS  OF  THE  STUDY 

This  study  provides  insight  into  critical  factors  and  their  threshold  values  for 
decision  makers  to  take  under  consideration  in  determining  how  many  merchant  ships  a 
warship  can  escort  and  what  scheme  should  be  followed. 

E.  METHODOLOGY 

The  scenario  described  is  modeled  using  the  MAN  A  modeling  environment. 

The  principles  of  robust  design  are  used  for  the  experiment.  This  research  also 
makes  use  of  the  newest  developments  in  Nearly  Orthogonal  Nearly  Balanced  Mixed 
designs  (NONBMD). 

Two  Nearly  Orthogonal,  Nearly  Balanced,  Mixed  designs,  one  for  decision 
factors  and  one  for  noise  factors,  are  generated  and  crossed  to  produce  the  overall 
experiment  design. 

The  experiment  is  performed  on  the  SEED  Center’s  high-performance  cluster, 
“Reaper.”  The  xstudy,  OldMcData,  and  Condor  software  packages  are  used  to  generate 
the  required  files  and  manage  the  execution  of  distributed  jobs. 

To  analyze  the  output  of  the  experiment,  the  following  methods  are  used: 
statistical  summaries,  multiple  regression,  partition  trees,  plots/graphs,  and  robust 
analysis. 
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F.  THESIS  ORGANIZATION 

Chapter  II  offers  an  outline  of  piracy  in  the  GOA  and  a  description  of  the 
Internationally  Recommended  Transit  Corridor  (IRTC).  The  chapter  ends  with  a 
description  of  the  European  Union  Naval  Force  mission  in  the  GOA.  Chapter  III  begins 
with  an  overview  of  the  modeling  tool  MANA  and  the  scenario  description  and  closes 
with  a  detailed  description  of  the  model  and  the  way  it  is  built.  Chapter  IV  includes  a 
discussion  of  experiment  design,  a  description  of  the  variables  used  in  the  analysis,  and 
an  explanation  of  NONBMD.  Chapter  V  describes  the  analytical  methods  used  to 
interpret  the  results  of  the  simulated  tests  and  concludes  with  an  explanation  of  the 
analytical  results.  Chapter  VI  is  a  discussion  of  the  results  and  recommendations  for 
follow-on  research. 
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II.  BACKGROUND 


A.  THE  HORN  OF  AFRICA  AND  GULF  OF  ADEN 

The  Horn  of  Africa  (Figure  2),  a  peninsula  in  East  Africa,  is  a  region  comprising 
the  countries  of  Eritrea,  Djibouti,  Ethiopia,  and  Somalia  and  is  laved  by  the  south  part  of 
the  Red  Sea  and  the  GOA  (Stock,  2004). 

The  Horn  of  Africa  is  one  of  the  most  strategically  important  international 
waterways.  It  carries  almost  the  95%  of  European  Union  (EU)  trade  (by  volume) 
transported  by  sea  and  20%  of  global  trade  (EUNAVFOR,  2010). 

Piracy  and  absence  of  the  rule  of  law  in  war-tom  Somalia  are  directly  linked.  In 
the  early  21st  century,  when  the  second  phase  of  the  Somali  civil  war  began,  piracy  off 
the  long  coastline  of  Somalia  began  to  be  a  threat  to  international  shipping.  The  lack  of  a 
substantial  government,  the  disintegration  of  the  Somali  armed  forces,  and  the  possibility 
of  gaining  quick  wealth  through  ransom  caused  a  rapid  increase  in  pirate  attacks  and 
armed  robbery  in  the  GOA  and  the  Somali  Basin  region  (SBR)  (UN  Chronicle,  n.d.). 

Some  observers  also  allege  that  illegal  over-fishing  of  the  Somalian  waters  by 
foreign  poachers  and  the  dumping  of  toxic  waste  have  motivated  some  Somali  groups 
engaged  in  piracy  (Berlatsky,  2010).  Big  trawlers  taking  advantage  of  the  lack  of 
Somalian  authority  were  illegally  fishing  vast  amounts  of  sea  life,  estimated  at  $300 
million’s  worth,  leaving  local  fishermen  with  empty  nets  (Merchant,  2009).  Also,  it  is 
hinted  that  a  large  amount  of  nuclear  waste,  including  lead,  cadmium,  and  mercury  has 
been  dumped  in  Somalian  waters  by  other  nations  and  individuals.  This  is  allegedly 
contributing  to  the  radical  decrease  in  sea  life  (Hari,  2009).  Early  on,  Somali  fishermen 
started  demanding  that  poachers  pay  fines,  giving  their  groups  names  such  as  the 
National  Volunteer  Coast  Guard  of  Somalia  or  Somali  Marines.  Later,  they  realized  there 
was  more  money  to  be  made  from  straightforward  abductions  for  ransom  (Berlatsky, 
2010). 
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Figure  2.  Horn  of  Africa  and  Gulf  of  Aden  (From  Herff  Jones  Nystrom,  2010) 

B.  PIRACY  IN  THE  GULF  OF  ADEN 

The  augmented  commercial  traffic  that  passes  through  the  straits  is  one  of  the 
main  reasons  for  the  increase  in  contemporary  piracy.  The  low  speed  required  for  safe 
passage  through  these  maritime  choke  points  makes  them  vulnerable  and  easy  targets 
(Chalk,  2008).  The  GOA  is  one  of  these  choke  points. 

The  GOA  and  area  surrounding  the  Horn  of  Africa  are  the  most  dangerous  areas 
for  piracy  in  the  world.  The  pirates  are  opportunity-driven  individuals,  divided  into  clans 
and  based  in  mobile  port  towns,  with  varying  patterns  of  operation  and  capabilities 
(Nelson,  2010).  They  are  usually  divided  into  groups  of  about  ten  people  and  use  small, 
high-speed  skiffs  towed  by  a  dhow,  pretending  to  be  fishermen.  The  skiffs  are  usually  in 
very  poor  condition,  old,  made  of  wood  or  fiberglass,  and  powered  by  outboard  motors; 
the  GOA’s  calm  seas  allow  the  use  of  such  vessels.  The  pirates  are  equipped  with 
drinking  water,  gasoline,  grappling  hooks,  short  ladders,  and  fishing  lines  and  nets.  They 
carry  a  large  variety  of  handheld  small  arms,  including  AK-47  rifles  and  rocket-propelled 
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grenades  (RPGs)  (Nelson,  2010)  and  (Kaplan,  2008).  They  venture  out  for  a  long  period, 
usually  three  weeks,  eating  raw  fish  while  waiting  for  their  potential  target.  When  they 
find  it,  they  use  ammunition  firing  to  coerce  the  master  of  the  vessel  to  either  slow  down 
or  stop,  and  then  climb  aboard,  making  for  the  bridge  to  take  control  of  the  ship.  After 
sailing  to  a  safe  harbor,  they  conduct  negotiations  for  ransom  while  crew  and  cargo  are 
held  for  weeks  (Kaplan,  2008). 


Figure  3.  International  Maritime  Bureau  Piracy  Map  for  2010  (From  International 

Chamber  of  Commerce,  2010) 

C.  INTERNATIONALLY  RECOMMENDED  TRANSIT  CORRIDOR 

Shared  Awareness  and  De-Confliction  (SHADE)  is  a  staff-level  working  group  of 
officers  from  the  various  operational  headquarters  of  all  the  alliances  and  individual 
countries  conducting  counter-piracy  operations.  One  of  the  products  of  this  group  is  the 
establishment  of  the  IRTC  (Nelson,  2010).  In  February  2009,  the  IRTC  replaced  the 
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previous  Maritime  Security  Patrol  Area  (MSPA)  (Figure  4)  in  order  to  introduce  safer 
and  more  organized  passage  for  all  merchant  vessels  transiting  the  GOA  (Rietveld,  2009). 

The  IRTC  is  a  corridor  between  Somalia  and  Yemen  within  international  waters, 
consisting  of  two  lanes,  each  of  five  nautical  miles  (nm’s)  width,  one  eastbound  and  one 
westbound,  with  a  space  of  two  nm  between  them.  The  total  length  of  the  transit  corridor 
is  480  nm,  and  a  vessel  maintaining  14  knots  requires  34.5  hours  to  pass  through  it. 
(North,  2010). 

Vessels  that  intend  to  pass  through  the  IRTC  are  grouped  according  to  transit 
speed.  Groups  of  10,  12,  14,  16,  18,  and  20  knots  transit  speed  are  formed  at  specific 
hours  and  dates,  and  traverse  the  IRTC  under  the  coalition’s  escort/convoy  services 
(North,  2010). 


Figure  4.  Internationally  Recommended  Transit  Corridor  (From 
http://www.eurocean.org/np4/file/863/Piracy  Gulf  of  Aden  Indian  Ocean.pd 

f,  2010) 
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D.  EUROPEAN  UNION  NAVAL  FORCE 


Due  to  the  importance  of  the  GOA  as  a  critical  conduit  for  international  trade, 
military  counter-piracy  operations  are  conducted  by  naval  ships  from  the  Combined  Task 
Force  150  (CTF-150),  NATO  Maritime  Group  (NMG),  Russia  (Russian  Maritime 
Security  Service  (RMSS)),  China,  Japan  (Japanese  Maritime  Self  Defense  Force 
(JMSDF)),  and  India.  Some  of  these  forces  operate  within  alliances  such  as  NATO  and 
EU,  some  as  part  of  the  CTF,  and  some  independently  represent  their  nation’s  interests. 
The  European  Union  Naval  Force  Somalia  (EUNAVFOR)  -  Operation  ATALANTA,  is 
the  European  Union’s  first  naval  operation,  in  support  of  resolutions  1814  (2008),  1816 
(2008),  1838  (2008)  and  1846  (2008)  of  the  United  Nations  Security  Council 
(EUNAVFOR,  2010). 

The  area  of  operations  of  EUNAVFOR  consists  of  the  south  Red  Sea,  the  Gulf  of 
Aden,  and  part  of  the  Indian  Ocean.  This  now  includes  the  Seychelles,  which  makes  the 
area  as  large  as  the  Mediterranean  Sea. 

The  aim  of  operation  ATALANTA  is  to  contribute  to: 

•  The  protection  of  the  vessels  of  the  World  Food  Program,  humanitarian 
aid,  and  the  African  Union  Military  Mission  in  Somalia  (AMISOM), 

•  The  protection  of  vulnerable  shipping, 

•  The  deterrence,  prevention  and  repression  of  pirate  activity  and  armed 
robbery,  and 

•  The  monitoring  of  fishing  activities  off  the  coast  of  Somalia. 

In  order  to  focus  the  scope  of  the  operation,  EUNAVFOR  has  been  granted  the 
authority  to  arrest,  detain,  and  transfer  persons  who  are  suspected  of,  or  who  have 
committed,  acts  of  piracy  or  armed  robbery  inside  the  area  of  operations.  They  can  also 
seize  the  vessels  of  the  pirates  and  the  vessels  hijacked  by  them,  as  well  as  the  goods  on 
board  (EUNAVFOR,  2010). 
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Figure  5.  An  Escorted  WFP  Ship  (From  http://www.eunavfor.eu/about-us/mission/, 

2010) 


10 


III.  MODEL  DEVELOPMENT 


A.  WHY  MODELS  ARE  USEFUL 

The  main  purpose  of  models  is  to  collect  information  and  gain  insight  about  some 
aspect  of  the  real  world.  A  model  mimics  or  describes  the  behavior  of  the  system  being 
modeled  (Sanchez  P.  J.,  2007).  Pedgen  (1995)  describes  the  use  of  models  in  simulation 
as  “[...]  the  purpose  of  understanding  the  behaviour  of  the  system  and/or  evaluating 
various  strategies  for  the  operating  system.” 

More  specifically,  models  provide  the  ability  to  explore  systems  that  cannot  be 
disturbed  because  they  are  either  sensitive  or  critical  (Chung,  2004).  Another  benefit  of 
the  model  is  its  ability  to  compress  time.  Systems  that  under  normal  conditions  need  a 
long  time  to  be  explored  can  effectively  be  shrunk.  Also,  in  some  cases,  the  cost  of 
building  a  model  is  less  expensive  than  experimenting  directly  with  a  real  system 
(Sanchez  P.  J„  2007). 

There  are  two  different  but  very  closely  connected  levels  of  modeling.  The  first 
level  is  the  simulation  model  itself;  it  is  the  model  that  describes  the  phenomena  of 
interest.  The  subsequent  level  is  a  statistical  model  that  is  built  to  describe  the 
simulation’s  responses.  The  latter  is  a  model  of  the  model,  and  is  often  referred  as  a 
metamodel  (Sanchez,  Lucas,  Sanchez,  Nannini,  &  Wan,  2011,  forthcoming). 

B.  MAP-AWARE,  NONUNIFORM  AUTOMATA  (MANA) 

1.  Overview  of  MANA 

Many  complex  systems  can  be  studied  through  the  use  of  agent-based  models. 
This  kind  of  model  provides  useful  insights  into  how  individual  interactions  give  rise  to 
emergent  properties  like  structures  and  causality.  Agent-based  modeling  is  a  useful  tool 
in  simulation  because  it  can  explore  phenomena  that  deal  with  interactions  between 
individual  agents,  for  example,  people  in  a  football  stadium  or  ants  searching  for  food 
(Berryman,  2008). 

The  model  chosen  for  this  thesis  is  one  such  model:  MANA,  Version  V.  MANA 
is  an  agent-based,  distillation-modeling  environment,  that  is,  it  attempts  to  create  models 
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that  capture  only  the  important  factors  in  a  situation  and  avoids  explicitly  all  physical 
details  (Home  &  Schwierz,  Data  farming  around  the  world  overview,  2008).  MANA  was 
developed  by  the  New  Zealand  Defense  Technology  Agency  (DTA)  for  use  as  a 
scenario-exploring  model  and  is  designed  to  address  a  broad  range  of  problems.  It 
resulted  from  the  frustrations  DTA  faced  in  analyzing  the  output  of  physics-based  models 
available  at  the  time,  basically  CAEn  (Close  Action  Environment)  and  Janus.  Although 
these  models  are  detailed,  highly  physics  based,  and  can  predetermine  the  behavioral 
outcomes  of  agents,  they  are  quite  limited  in  analysis,  basically  because  it  is  difficult  to 
get  the  entities  to  behave  in  unscripted  ways  (McIntosh,  Galligan,  Anderson,  &  Lauren, 
2007). 

In  MANA,  a  realistic  scenario  can  be  taken  and  a  rough  model  quickly  built.  That 
is  because  MANA  is  not  intended  to  describe  every  aspect  of  a  military  operation.  Once 
the  skeleton  of  the  model  has  been  built,  one  can  easily  change  agent  parameters  and 
states  in  order  to  create  a  reasonably  accurate  model  of  the  desired  interactions. 

Two  of  MANA’s  limitations  are  that  is  has  not  been  designed  to  examine  careful 
formation  fighting,  and  entities  do  not  always  behave  in  a  sensible  manner  (McIntosh, 
Galligan,  Anderson,  &  Lauren,  2007).  This  is  also  the  major  advantage  of  MANA, 
meaning  that  the  lack  of  a  central,  predetermined,  decision-making  algorithm  for 
controlling  entity  behavior  results  in  the  agents  making  their  own  decisions  as  they  adapt 
to  the  environment  (Walton,  Paulo,  McCarthy,  &  Vaidyanathan,  2005). 

Finally,  MANA  is  supplied  with  a  built-in  random-number  generator  using  the 

on 

Delphi  function  “random.”  The  random-number  generator  has  a  cycle  of  2  and 
maintains  a  32-bit  seed.  This  means  that  the  result  can  take  on  approximately  4.3  x  109 
values  before  repeating  the  cycle. 

2.  Characteristics  of  MANA 

MANA  was  adopted  from  the  ISAAC  model,  developed  by  the  Center  for  Naval 
Analyses,  in  order  to  explore  key  concepts  that  the  ISAAC  model  was  not  able  to  handle 
(McIntosh,  Galligan,  Anderson,  &  Lauren,  2007).  Those  concepts  are: 
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•  Situational  awareness:  MANA  provides  two  types  of  situational 
awareness  maps:  a  squad  map  that  holds  direct  squad  contacts  and  an 
inorganic  map  that  stores  contacts  provided  by  other  squads  through 
communication  links. 

•  Communications:  Allows  squads  to  communicate  with  other  squads  in 
case  of  contact  sightings. 

•  Terrain  map:  Contains  terrain  features  such  as  roads  that  can  be  followed 
by  agents  and  underbrush  that  can  be  used  as  concealment. 

•  Waypoints:  Can  define  a  set  of  waypoints,  not  merely  the  final  goal. 

•  Event-driven  personality  changes:  Different  events  such  as  contact  of  an 
enemy  can  change  the  behavior  of  an  agent  or  whole  squad.  The  new 
behavior  lasts  for  a  user-specified  period. 

MANA  has  a  well-developed  graphical  user  interface  (GUI),  and  although  it  was 
originally  developed  to  model  land  warfare,  it  can  model  alternative  scenarios,  such  as 
sea  based  (McIntosh,  Galligan,  Anderson,  &  Lauren,  2007);  see,  for  example,  Abel 
(2009). 

C.  MODEL  DEVELOPMENT 

1.  Scenario  Description 

As  explained,  the  IRTC  was  established  so  that  merchant  or  other  vessels  can 
assemble  at  specific  points  and  transit  as  a  group  under  the  escort  of  coalition  warships 
(Figure  6). 


Figure  6.  Warships  Escorting  Merchant  Vessels  in  IRTC  (After 

http://asianyachting.com/news/PirateCorridor.htm,  2011) 
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The  scenario  shown  in  Figure  7  takes  place  in  the  GOA  and  specifically  in  the 
IRTC.  It  simulates  coalition  operations  to  protect  a  convoy  of  merchant  ships  transiting 
westbound  along  the  IRTC.  Usually  in  the  area  there  are  also  fishing  or  other  individual 
traveling  vessels,  and  the  pirates  behave  like  one  of  them  until  they  approach  a  potential 
target.  The  scenario  commences  from  the  point  that  the  pirates  have  reached  high  velocity 
heading  towards  the  convoy.  The  pirates  can  be  identified  by  one  of  the  merchant  ships  or 
by  the  escorting  warship. 

The  merchant  ships  are  sailing  in  two-column  formation.  As  soon  as  one  realizes 
it  is  being  approached  by  a  quickly  moving,  suspicious  vessel,  it  signals  the  fleet, 
increases  speed,  and  begins  evasive  tactics  while  the  entire  convoy  flees.  The  warship, 
which  may  be  sailing  beside,  before,  or  behind  the  merchant  formation,  increases  speed 
and  heads  towards  the  merchant  under  attack.  At  the  same  time,  orders  are  given  for  a 
helicopter  to  take  off,  which  heads  towards  the  provocation. 


Figure  7.  Map  of  the  Scenario 
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2. 


Measure  of  Effectiveness 


As  a  measure  of  effectiveness  (MOE),  the  mean  merchant  (blue)  casualties  are 
computed.  This  is  easily  assessed  using  JMP  by  summarizing  over  the  number  of  blue 
casualties  for  each  design  point  of  the  experiment. 

3.  Scenario  Assumptions 

Any  scenario  must  have  some  assumptions  if  it  is  to  be  framed  in  terms  of  time 
and  space.  For  the  purpose  of  this  study,  the  key  assumptions  are: 

•  The  pirate  reveals  his  identity  when  he  is  close  to  the  potential  target.  This 
proximity  will  always  be  within  sensor  range.  Therefore,  the  quality  of  the 
sensors  on  the  warship  and  the  cargo  ships  has  no  impact  on  pirate 
detection. 

•  The  helicopter  and  the  warship  cannot  be  attacked  or  fired  on  by  the 
pirates. 

•  In  the  case  of  multiple  pirates,  either  the  helicopter  or  the  warship  will 
interdict  the  pirates  by  firing  ammunition  against  them  instead  of  trying  to 
stop  and  arrest  them.  Thus,  the  helicopter  or  the  warship  can  help  in 
pursuing  the  other  pirates. 

•  In  representing  the  capture  of  a  merchant  ship,  the  range  of  the  pirate 
weapon  is  limited  to  75  meters  and  the  merchant  needs  to  be  shot  twice. 

•  The  merchant  ships  are  sailing  in  two-column  formation. 

4.  Model  Description 
a.  Battlefield 

The  battlefield  is  a  20  x  20  nm  snapshot  of  the  IRTC.  There  are  no 
battlefield  restrictions,  as  the  area  is  open  sea.  One  model  time-step  is  set  to  equal  ten 
seconds.  To  configure  the  settings  of  the  battlefield,  the  option  Edit  Battlefield  has  been 
used  in  the  setup  option  in  MANA’s  menu  bar.  The  battlefield  settings  of  the  scenario  are 
shown  in  Figure  8.  These  settings  are  found  on  MANA’s  Setup  menu  on  the  Edit 
Battlefield  screen.  All  other  selections  are  at  their  default  values. 
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Figure  8.  MANA  Battlefield  Settings 
b.  Squads 

The  squad  is  a  key  concept  in  MANA.  A  squad  is  a  group  of  initially 
homogeneous  agents  given  a  size  between  0  and  1000,  as  defined  by  the  user.  Table  1 
shows  the  squads  used  in  this  study. 
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Squad  # 

Unit 

Allegiance 

1 

Warship 

1  (BLUE) 

2 

Merchant  ships 

1  (BLUE) 

3 

Pirates 

2  (RED) 

4 

Helicopter 

1  (BLUE) 

5 

Fishermen 

0  (NEUTRAL) 

Table  1.  Model  Squads 


c.  Warship 

The  warship  squad  consists  of  a  single  agent.  The  agent’s  personality 
weightings  and  different  trigger  states  are  shown  in  Figure  9.  The  personality  weightings 
are  defined  using  slide  bars  and  may  take  on  values  from  -100  to  100.  A  positive  value 
indicates  a  positive  propensity  for  the  associated  personality  weighting,  while  a  negative 
value  indicates  a  negative  propensity.  Any  propensity  that  has  been  changed  from  the 
default  zero  is  shown  in  red.  The  warship,  after  contacting  the  pirates,  has  a  propensity  to 
move  towards  them  (the  Enemies  weighting  is  set  to  50).  All  other  selections  are  at  their 
default  values. 

States  that  are  being  used  for  the  warship  (other  than  the  default  state),  are 
Enemy  Contact,  for  behavior  when  the  warship  contacts  the  pirates  using  its  own  sensors, 
Inorganic  SA  Enemy  Contact  2,  when  the  warship  is  notified  by  the  merchants  using 
communication  links,  and  Spare  1 ,  for  releasing  the  helicopter  after  enemy  contact. 
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Figure  9.  Warship  Squad  Properties 


The  Inorganic  SA  panel  controls  the  flow  of  situational  awareness  among 
squads  by  using  communication  links  as  conduits  of  information.  The  parameters  of  this 
panel  do  not  vary  within  the  squad’s  different  trigger  states.  The  warship  has 
communication  links  with  the  merchants  and  helicopter,  as  shown  in  Figure  10,  Inorganic 
SA  represents  the  actual  communication  between  the  warship,  merchants,  and  helicopter. 
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Figure  10.  Warship  Inorganic  SA  Communications 


Each  squad  has  up  to  four  weapons  available,  and  most  of  the  parameters 
for  the  weapons  are  able  to  change  within  a  trigger  state.  The  warship  has  only  one 
weapon,  set  to  fire  within  a  range  of  250  meters  and  used  to  represent  pirate  deterrence. 
In  the  Protect  Contact  Type  box,  Self  and  Other  Friends  are  selected  to  prevent  the 
warship  from  firing  against  itself  or  the  merchant  ships.  All  other  selections  are  at  default 
values.  The  warship’s  weapon  adjustments  are  shown  in  Figure  11. 
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Figure  1 1 .  Warship  Weapons 
d.  Merchant  Ships 

The  merchant  squad  consists  of  eight  agents.  The  agents’  personality 
weightings  and  different  trigger  states  are  shown  in  Figure  12.  The  merchants,  after 
contact  with  the  pirates,  have  a  propensity  to  move  away  from  them  (the  Enemies 
weighting  is  set  to  -30).  All  other  selections  are  at  their  default  values. 

The  state  used  for  the  merchant  ships,  other  than  the  default,  is  Enemy 
Contact,  for  behavior  after  contacting  the  pirates. 
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Figure  12.  Merchant  Ships  Squad  Properties 
e.  Pirates 

The  pirate  squad  consists  of  one  or  more  agent.  Agent  personality 
weightings  and  different  trigger  states  are  shown  in  13.  The  pirates,  after  contact  with  the 
convoy,  have  a  propensity  to  move  towards  the  merchants  (Enemy  Threat  3  and  Ideal 
Enemy  weighting  are  set  to  100)  while  avoiding  the  warship  and  helicopter  (Enemy 
Threat  1  weighting  set  to  -100).  All  other  selections  are  at  their  defaults. 


Other  than  the  default,  the  state  used  for  the  pirates  is  “Enemy  Contact  3,” 
for  behavior  after  the  pirates  contact  the  convoy. 
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Figure  13.  Pirates  Squad  Properties 

The  pirates,  like  the  warship,  have  only  one  weapon.  Their  weapon  is  set 
to  fire  at  a  range  of  75  meters,  and  is  used  to  represent  a  merchant  ship’s  capture.  Using 
the  Fire  on  Targets  in  This  Class  Order  input  box,  the  pirates  are  assigned  to  fire  against 
the  merchant  ships  only,  while  the  use  of  Non  Target  Classes  keeps  them  from  firing 
against  the  warship  and  helicopter.  All  other  selections  are  at  their  default  values.  The 
pirates’  weapon  adjustments  are  shown  in  Figure  14. 
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Figure  14.  Pirates  Weapons 
/.  Helicopter 

The  helicopter  squad  consists  of  one  agent.  The  agent’s  personality 
weightings  and  different  trigger  states  are  shown  in  Figure  15.  The  helicopter,  after 
release  from  the  warship,  has  a  propensity  to  move  towards  the  pirates  (the  Enemies  and 
Enemy  Threat  2  weightings  are  set  to  100  and  40,  respectively).  To  ensure  that  the 
helicopter  will  not  be  hit  by  friendly  or  enemy  fire,  the  Easy  Going  and  Concealment 
weightings  have  been  set  to  100.  All  other  selections  are  at  default  values. 


The  state  that  is  used  for  the  helicopter,  other  than  the  default,  is  Released 
From  Embuss,  for  behavior  after  being  released  from  the  warship. 
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Figure  15.  Helicopter  Squad  Properties 


The  helicopter  has  only  one  weapon.  It  is  set  to  fire  at  a  range  of  250,  200, 
and  100  meters,  with  an  assigned  probability  of  0.3,  0.5,  and  1,  respectively.  It  represents 
pirate  deterrence.  In  the  Protect  Contact  Type  box,  Self  and  Other  Friends  are  selected  to 
prevent  the  helicopter  from  firing  against  itself,  the  warship,  or  the  merchant  ships.  All 
other  selections  are  at  their  default  values.  The  helicopter’s  weapon  adjustments  are 
shown  in  Figure  16. 
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Figure  16.  Helicopter  Weapons 


g.  Fishermen 

The  fishermen  squad  motion  does  not  influence  the  development  of  the 
scenario.  The  agent’s  personality  weightings  are  shown  in  Figure  17.  The  Next  Waypoint 
weighting  is  set  to  20  to  represent  the  traffic  in  the  IRTC. 
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Figure  17.  Fishermen  Squad  Properties 


h.  Stop  Conditions 

The  Stop  Conditions  menu  in  MANA  contains  several  ways  to  terminate  a 
run,  namely,  having  either  agent  reach  his  waypoints  or  specifying  the  number  of  losses 
sustained  by  one  side,  or  both.  These  settings  are  found  on  MANA’s  pull-down  Setup 
menu  on  the  Stop  Conditions  screen.  A  scenario  terminates,  as  shown  in  Figure  18,  when 
the  red  or  blue  squads  have  one  loss  or  when  the  merchant’s  ships  (Squad  2)  reach  the 
waypoint.  Where  the  number  of  pirates  is  greater  than  one,  the  stop  conditions  for  the  red 
losses  are  changing  accordingly. 
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Figure  18.  Scenario  Stop  Conditions 
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IV.  DESIGN  OF  EXPERIMENTS  (DOE) 


A.  DATA  FARMING 

One  tool  used  in  this  project  is  data  farming.  Data  farming  can  be  combined  with 
the  use  of  distillation  models  like  MANA  to  explore  a  more  complete  landscape,  i.e., 
more  possible  system  outcomes,  than  merely  trying  to  answer  our  problem  (Home  & 
Schwierz,  Data  farming  around  the  world  overview,  2008). 

Any  model  can  be  explored  using  data  farming,  but  the  use  of  distillations  has  the 
advantage  that  the  model  can  be  developed  in  a  very  short  time,  from  a  few  minutes  to 
hours  or  days.  Once  a  model  has  been  developed,  high-performance  computing  (HPC),  in 
combination  with  one  of  several  existing  DOE’s,  can  be  used,  and  a  huge  sample  space 
can  be  efficiently  and  rapidly  explored.  Simulations  are  run  thousands  of  times  while  the 
input  parameters  are  varied  across  a  wide  value  space. 

The  idea  behind  this  technique  is  that  it  provides  better  insight  into  the  question 
and  reveals  potential  surprises,  both  positive  and  negative,  that  can  lead  to  better- 
informed  decisions  (Horne  &  Meyer,  Data  farming:  discovering  surprise,  2004),  (Horne 
&  Schwierz,  Data  farming  around  the  world  overview,  2008). 

B.  DESIGN  FACTORS 

In  a  design,  it  is  fundamental  to  identify  all  the  factors  that  are  expected  to  affect 
the  system  response.  The  factors  involved  here  are  classified  as  decision,  noise,  or 
artificial  factors.  Decision,  or  controllable,  factors  are  those  we  can  control  in  the  real 
world,  while  noise,  or  uncontrollable,  factors  are  those  we  cannot  readily  control  in  real 
life,  or  only  at  great  expense.  Finally,  artificial  factors  are  those  that  come  from  the 
simulation  itself,  e.g.,  the  initial  state  of  the  system,  the  warm-up  period,  or  terminal 
conditions  (Sanchez  S.  M.,  Robust  design:  seeking  the  best  of  all  possible  worlds,  2000). 

1.  Decision  Factors 

The  following  factors  have  been  chosen  as  controllable  factors: 
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a.  Warship  Position 

This  is  a  categorical  variable  with  three  levels  for  the  position  of  the 
warship  in  reference  to  the  merchant  ships:  “behind”  (indicated  as  1),  “on  their 
sides/flanks”  (indicated  as  2),  and  “in  front”  (indicated  as  3). 

b.  Convoy  Speed 

This  is  a  continuous  variable,  sampled  in  discrete  values,  for  the  normal 
traveling  speed  of  the  convoy  (warship  and  merchants).  It  is  varied  between  10  and  20 
knots. 


c.  Number  of  Merchant  Ships 

This  is  a  discrete  variable  for  the  number  of  merchant  ships  in  the  convoy. 
It  is  varied  between  10  and  30. 

d.  Distance  Between  Merchant  Ships 

This  is  a  continuous  variable,  sampled  in  discrete  values,  for  the  distance 
between  merchant  ships  while  in  transit.  It  is  varied  between  50  and  500  meters. 

e.  Helo  Take  Off  Delay 

This  is  a  continuous  variable,  sampled  in  discrete  values,  for  the  time  it 
takes  the  helicopter  to  lift  off  after  the  warship  becomes  aware  of  the  pirates,  whether 
alerted  by  its  own  sensors  or  a  merchant  ship’s.  It  is  varied  between  15  and  60  minutes. 

2.  Noise  Factors 

The  following  factors  have  been  chosen  as  uncontrollable  factors: 

a.  Number  of  Pirates 

This  is  a  discrete  variable  for  the  number  of  the  pirates  that  can  attack  the 
convoy.  It  is  varied  between  1  and  4. 
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b.  Communication  Latency  Between  Merchant  Ships  and  Warship 

This  is  a  continuous  variable,  sampled  in  discrete  values,  for  the  latency  in 
communication  between  the  merchant  ship  under  attack  and  the  warship.  It  is  varied 
between  1  and  10  minutes. 

c.  Pirates  Speed 

This  is  a  continuous  variable,  sampled  in  discrete  values,  for  the  speed  of 
pirate  vessels.  It  is  varied  between  25  and  45  knots. 

d.  Distance  from  Convoy  That  Pirates  Reveal  Identity 

This  is  a  continuous  variable,  sampled  in  discrete  values,  for  the  distance 
from  the  merchant  ships  at  which  the  pirates  reveal  their  identity  and  increase  speed.  It  is 
varied  between  3000  and  5500  meters. 

C.  EXPERIMENTAL  DESIGNS 

The  ultimate  purpose  of  a  simulation  model  is  to  gain  insight  into  questions.  With 
a  well-designed  experiment,  the  analyst  is  able  to  explore  more  factors  than  would 
otherwise  have  been  possible  by  using  DOE  techniques  developed  specifically  for 
analyzing  high-dimensional  computational  models  (Sanchez,  Lucas,  Sanchez,  Nannini,  & 
Wan,  forthcoming). 

Which  DOE  is  suitable?  Although  many  designs  are  available,  the  choice  involves 
several  issues,  such  as  the  complexity  of  the  response,  the  time  required  for  the 
simulation  to  run,  and  the  ease  of  changing  the  parameter  values  (Kleijnen,  Sanchez, 
Lucas,  &  Cioppa,  2005). 

One  possibility  is  the  classical  full-factorial  (or  gridded)  design.  Full-factorial 
designs  are  orthogonal  designs  in  which  the  pairwise  correlation  between  any  two 
columns  is  zero.  Though  full-factorial  designs  examine  all  the  possible  combinations  of 
factor  levels,  nevertheless,  for  studies  involving  a  moderate  or  large  number  of  factors, 
they  are  extremely  large,  and  thus  unmanageable.  (Vieira,  Sanchez,  Kienitz,  & 
Belderrain,  2011) 
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Another  possible  design  is  the  fractional-factorial  design,  which  can  cover  only 
two  (2k  factorial  design)  or  more  (mk  factorial  design)  levels  of  each  factor.  They  are  also 
orthogonal  designs  and  they  let  us  examine  more  than  one  factor  at  a  time.  Additionally, 
for  the  later  designs,  the  larger  the  value  of  m,  the  better  space-filling  it  has.  However, 
fractional-factorial  designs  are  inefficient  when  k  is  large  (Sanchez  S.  M.,  Better  than  a 
petaflop:  the  power  of  efficient  experimental  design,  2008). 

A  third  option  for  experimental  design  is  the  Latin  hypercube  (LH).  LHs  have 
good  space-filling  properties,  like  mk  factorial  designs,  but  require  fewer  samples.  Unlike 
the  2k  factorial,  they  also  provide  information  about  what  is  happening  in  the  center  of  the 
experimental  region  (Sanchez  S.  M.,  Better  than  a  petaflop:  the  power  of  efficient 
experimental  design,  2008). 

Finally,  Cioppa  and  Lucas  (2007)  introduced  Nearly  Orthogonal  Latin  Hypercube 
(NOLH)  designs.  These  designs  have  nearly  orthogonal  properties  (pairwise  correlations 
less  than  0.05),  good  space-filling,  and  can  accommodate  a  large  number  of  factors.  A 
design  that  is  nearly  orthogonal  will  not  suffer  effects  due  to  adverse/multicollineatity 
(Montgomery,  Peck,  and  Vining,  Introduction  to  linear  regression  analysis,  2006). 
NOLHs  deal  better  with  continuous  factors;  rounding  in  order  to  accommodate  discrete 
factors  has  a  result  of  increasing  correlation  and  thus  losing  some  of  the  nearly 
orthogonal  properties  (Sanchez  S.  M.,  Better  than  a  petaflop:  the  power  of  efficient 
experimental  design,  2008).  Figure  19  shows  the  space-filling  of  selected  designs. 


Figure  19.  Scatterplot  Matrices  for  Selected  Factorial  and  NOHL  Designs.  (From 

Sanchez,  2008) 
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D.  NEARLY  ORTHOGONAL  NEARLY  BALANCED  MIXED  DESIGN 


For  this  study,  the  NONBMD  is  used.  NONBMD  developed  by  Helcio  Vieira 
(2010)  for  his  PhD  (Figure  20),  are  designs  suitable  for  dealing  with  a  combination  of 
continuous,  categorical,  and  discrete  factors.  The  design  is  “nearly  orthogonal”  and 
“nearly  balanced,”  which  means  that  the  absolute  value  of  the  maximum  pairwise 
correlation  is  less  than  0.05,  and  that  it  samples  factors  with  different  numbers  of  levels 
in  a  balanced  manner  (Vieira,  Sanchez,  Kienitz,  &  Belderrain,  2011). 


CREATE 


Figure  20.  Snapshot  of  the  Nearly  Orthogonal,  Nearly  Balanced,  Design  Creator 

For  this  analysis,  two  separate  designs  were  created,  one  for  decision  factors  and 
one  for  noise.  The  first  design  provides  51  design  points  (DP),  while  the  second  provides 
forty-two  (Figure  21).  A  design  point  is  a  unique  combination  of  input  factors. 
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Figure  21.  Screen  Shots  of  the  Separate  Designs  for  Decision  and  Noise  Factors.  The 
display  shows  the  decision  factors,  the  noise  factors,  ranges,  levels,  and  factor 

classification. 


Both  the  decision  and  noises  factors,  as  shown  in  Figure  22,  have  good  space¬ 
filling  properties  while  the  maximum  pairwise  correlations  are  0.0016  and  zero, 
respectively. 


34 


Multivariate 

Correlations 


Multivariate 

Correlations 


warshipPos  convoySpeed  #ofMerch  distBetwMerch  heloDelay 


warshipPos 

1.0000 

-0.0000 

-0.0000 

-0.0000 

0.0000 

convoySpeed 

-0.0000 

1.0000 

-0.0000 

0.0004 

0.0000 

#ofMerch 

-0.0000 

-0.0000 

1.0000 

0.0014 

-0.0004 

distBetwMerch 

heloDelay 

-0.0000 

0.0000 

0.0004 

0.0000 

0.0014 

-0.0004 

-0.0016 
)  1.0000 

Scatterplot  Matrix 


2.5  - 

2  - 

1.5  - 

1  - 

warshipPos 

. 

..  ..\ 

20  - 

N 

. .  y 

/.  _ 

16  - 

convoySpeed 

■  ■  -■ 

-- 

12  - 

.... 

.. 

8  - 

V . 

> 

N "  ■ 

y 

'v'  '  " 

'■> 

/:  :  . 

.\ 

/..... 

\ 

/.  . 

■  A 

25  - 

20  - 

#ofMerch 

15  - 

. ■■ 

10  - 

\=  '  = 

'  '/ 

\  ' 

■  "v 

\ 

■'/ 

/=  .  ■ 

:\ 

N 

400  - 

'  ■  i 

I  ;  ; 

' ; 

300  - 

:  |  ■ 

;  1 

distBetwMerch 

:  /  ■' 

200  - 

j  ! 

.  \ 

■■■.  '■ 

100  - 

0  - 

:  ;  I 

\  ■  ' 

' :  y 

V  ' 

/ 

K'  '■ 

V 

.  > 

/f;  ■ 

: 

Z'  . 

X 

/A  - 

■;N 

50  - 

40  - 

S 

heloDelay 

30  - 

i  "  ' 

■  : 

20  - 

j 

i 

(  .  . 

* 

10  2 

1  ' 

V 

'  '  V 

v  ■  " 

V 

1  1.5  2  2.5  810  14  18  10152025  0100  300  1(20304050 


#ofPiratescomLatencyBetwShips  RedSpeed  DistRedReveal 


#ofPi  rates 

1.0000 

0.0000 

-0.0000 

0.0000 

comLatencyBetwShips 

0.0000 

1.0000 

0.0000 

-0.0000 

RedSpeed 

-0.0000 

0.0000 

1.0000 

0.0000 

DistRedReveal 

0.0000 

-0.0000 

0.0000 

1.0000 

Scatterplot  Matrix 


3.5  - 

2.5  - 

1.5  - 

0  5- 

#ofPirates 

/ . 

\ . 

...\ 

io : 

-\ 

8  - 

6  - 

4  - 

2  - 

0  - 

\  ■  "■ 

V 

s  '  / 

45  - 

y 

.  \ 

/;  _  n 

40  - 

35  - 

RedSpeed 

30  - 

25  - 

y  ■' 

y 

\  '  / 

5500  - 

* 

/ . 

.  \ 

5000  - 

4500  - 

DistRedReveal 

4000  - 

3500  - 

\ . 

7 

0.51.522.533.54  01  23456789  25  30  35  4045  3500  4500 


Figure  22.  Pairwise  Correlation  Matrix  for  Decision  Factors  (left)  and  Noise  Factors 

(right). 


Next,  the  two  separate  designs  are  crossed  using  a  ruby  program,  creating  a  total 
of  2142  DP  (51  x  42).  The  crossed  design  has  also  maximum  correlation  0.0016  and  good 
space-filling,  as  shown  in  Figure  23. 
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Figure  23.  Pairwise  Correlation  Matrix  for  the  Crossed  Design. 

Finally,  the  experiment  was  executed  on  the  Simulation  Experiments  and 
Efficient  Design  (SEED)  Center’s  high-performance  cluster,  “Reaper.”  A  hundred  and 
fifty  replications  of  each  DP  were  performed  for  data  farming,  yielding  a  total  number  of 
321,300  simulated  convoy  operations.  The  xstudy,  OldMcData,  and  Condor  software 
packages  were  used  to  generate  the  required  files  and  manage  the  execution  of  the 
distributed  jobs  (see  http://harvest.nps.edu).  Since  each  replication  completed  very 
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quickly  (in  seconds),  this  experimental  design  took  only  a  little  more  than  five  hours  to 
complete,  much  less  time  than  it  would  take  to  run  the  same  scenario  in  traditional  full- 
factorial  design  or  2m  design  (Figure  24). 


Figure  24.  Comparison  Plot  for  the  Time  Acquired  for  the  Experiment  to  Run 


E.  ROBUST  DESIGN 

Robust  design  is  a  technique  that  combines  Taguchi’s  philosophy  and  strategy  on 
improving  product  performance  (1980)  and  response-surface  metamodeling.  The  use  of 
robust  designs  and  analysis  may  lead  to  decisions  based  not  only  on  a  mean  system’s 
performance,  but  also  on  the  variance  of  its  performance.  Specifically,  the  system’s 
performance  must  be  acceptable  and  relatively  insensitive  to  uncontrollable  sources  that 
may  increase  its  variance  (Sanchez  S.  M.,  Robust  design:  seeking  the  best  of  all  possible 
worlds,  2000). 


37 


As  mentioned,  classifying  design  factors  into  decision  or  noise  is  fundamental.  In 
robust  design,  this  classification  is  important  for  the  analyst  to  identify  design  points  that 
yield  a  good  performance  of  a  system  across  the  range  of  all  noise  factors  (Sanchez  S. 
M.,  Better  than  a  petaflop:  the  power  of  efficient  experimental  design,  2008). 

The  analyst  specifies  a  target  value  x  and  evaluates  the  performance  of  an  MOE 
compared  to  that  value.  This  is  achieved  by  the  use  of  a  loss  function  such  as  a  quadratic, 
inverted  normal,  or  truncated  quadratic.  In  this  thesis,  a  quadratic  loss  function  is 
evaluated  to  minimize  blue-squad  casualties.  The  benefits  of  the  quadratic  loss  function 
are  that  it  is  easy  to  manipulate  mathematically  and  that  it  is  a  kind  of  loss  function  used 
in  many  other  statistical  applications,  like  regression. 

Evaluating  a  robust  design  may  yield  beneficial  results  for  the  system 
performance  under  examination.  First,  the  system  may  perform  well  across  a  range  of  all 
noise  factors,  resulting  in  fewer  surprises  when  moving  from  simulation  to  real  life 
implementation.  Another  benefit  is  improved  communication  between  analyst  and  client 
by  the  use  of  the  expected  loss.  Finally,  robust  designs  facilitate  continuous  improvement 
and  lead  to  better  decisions  (Sanchez  S.  M.,  Robust  design:  seeking  the  best  of  all 
possible  worlds,  2000). 

Having  crossed  the  decision  design  with  the  noise  design,  we  have  created  a 
design  that  serves  well  the  purpose  of  robust  analysis  and  we  focus  our  attention  on  blue 
casualties  (blueCas),  with  the  goal  of  minimizing  them. 

First,  we  introduced  the  raw  data  from  321,300  runs  of  MANA  in  JMP.  We  have 
specified  the  target  value  x  equal  to  zero  blue  casualties,  and  then  created  a  new  column 

for  the  loss,  making  use  of  the  quadratic  loss  function:  i  (blueCas)  =  (blueCas  -  x)  2. 

Finally,  we  summarized  over  the  noise  factors,  calculating  the  mean  loss  and  the  mean 
and  standard  deviation  of  the  response  blueCas. 
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V.  DATA  ANALYSIS 


A.  MULTIPLE  REGRESSION  ANALYSIS  /  INTERACTION  PLOTS:  ALL 
FACTORS 

Multiple-regression  analysis  is  a  common  statistical  technique  for  exploring  and 
modeling  the  relationships  between  various  factors,  called  predictor  or  regressor 
variables,  and  a  response  variable  (Montgomery,  Peck,  &  Vining,  2006).  Various 
statistical  packages  are  available  to  facilitate  regression  analysis.  JMP  Statistical 
Discovery  Software  version  9.0.0  is  used  for  this  work. 

The  coefficient  of  determination,  R-squared,  is  a  popular  way  to  assess  the  overall 
adequacy  of  a  model.  R-squared  shows  the  proportion  of  the  total  variation  of  the 
response  variable  that  is  explained  by  the  model.  It  takes  values  between  zero  and  one, 
with  the  higher  R-squared  values  (closer  to  one)  being  more  desirable.  However,  R- 
squared  never  decreases,  even  when  an  independent  variable  is  statistically  insignificant. 
Therefore,  an  analyst  must  always  be  aware  that  using  R-squared  may  result  in  an  over¬ 
fitted  model. 

To  avoid  over-fitting,  a  better  statistic  for  the  overall  adequacy  of  a  model  is  the 
adjusted  R-squared.  The  difference  between  R-squared  and  adjusted  R-squared  is  that  the 
former  can  go  down  if  statistically  insignificant  predictors  are  entered  into  a  model 
(Montgomery,  Peck,  &  Vining,  2006). 

Once  the  regression  model  is  created,  an  adequacy  check  should  be  performed. 
The  four  major  assumptions  made  about  the  residuals/errors  are  the  following: 

•  They  have  mean  zero. 

•  They  have  constant  variance. 

•  All  errors  are  independent. 

•  All  errors  are  normally  distributed. 

The  above  assumptions  can  be  abbreviated  as  s  ~  NID  (0,  a2)  (Montgomery,  Design  and 
analysis  of  experiments,  2009).  If  any  of  these  assumptions  are  violated,  the  analyst  has  a 
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variety  of  tools  for  resolving  the  issue.  For  example,  transformations  or  other  advanced 
techniques  can  be  used  if  normality  assumptions  are  violated.  (Montgomery,  Peck,  & 
Vining,  Introduction  to  linear  regression  analysis,  2006) 

The  plots  and  models  were  generated  using  JMP.  We  first  imported  321,300  rows 
of  raw  data,  indicated  in  the  previous  chapter,  in  JMP.  Next,  we  created  a  summary  data 
table  with  all  independent  variables  by  averaging  responses  of  the  two  MOEs,  mean  blue 
casualties  and  mean  red  casualties,  across  the  150  runs  of  each  DP.  This  new  data  table 
contained  2,142  rows. 

Analyzing  the  distribution  for  the  mean  blue  casualties  we  see  that  the  majority  of 
the  output  is  zero  casualties  with  a  mean  of  0.175  and  standard  deviation  of  0.18,  with 
maximum  casualties  of  0.78,  as  shown  in  Figure  25. 
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Figure  25.  Distribution  for  the  Mean  Blue  Casualties 


The  process  for  creating  the  regression  model  consisted  of  a  forward  Bayesian 
information-criterion  (BIC)  stepwise  technique.  We  allowed  main  effects,  two-way 
interactions,  and  second-order  polynomial  terms  into  the  model.  We  examined  the 
stepwise-regression  step  history  in  order  to  see  the  “knee  in  the  curve”  of  R-squared  as 
new  terms  were  added  to  the  model.  We  used  this  information  to  guide  us  in  final 
selection  of  our  regression  model.  As  Figure  26  indicates,  we  quickly  reach  a  point  where 
adding  statistically  significant  terms  has  essentially  no  practical  impact  on  the  fit  of  the 
model. 
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Figure  26.  R-Squared  Values  of  the  Fitted  Model  for  Mean  Blue  Casualties 


Figure  27  displays  the  actual  value  by  predicted  plot  and  summary  data  for  the 
final  model.  This  model,  consisting  of  13  terms,  including  the  intercept,  explains  almost 
94%  of  the  variance  within  the  model.  For  that  model,  the  adjusted  R-squared  is  0.93596. 


Figure  27.  Fitted  Model  for  Mean  Blue  Casualties 
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To  check  the  adequacy  of  the  model,  we  examined  whether  the  four  assumptions 
for  the  residuals  hold.  First,  the  independence  assumption  holds  because  we  used 
different  random-number  seeds.  The  mean  zero  and  constant  variance  assumptions  are 
also  satisfied,  as  shown  in  Figure  28. 


Row 


Figure  28.  Residual  by  Row  Plot 

The  only  assumption  that  is  not  met  is  the  normality  of  the  errors,  as  indicated  in 
Figure  29.  The  histogram  looks  fine,  but  the  normal  quantile  plot  and  the  goodness  of  fit 
test  justify  the  rejection  of  normality.  One  reason  for  this  rejection  is  that  we  are  dealing 
with  proportions  and  are  limited  to  more  zeros  than  ones.  Asymptotically,  we  expect  our 
distribution  to  be  closer  to  normal. 
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Figure  29.  Distribution  of  the  Residuals  for  the  Fitted  Model 
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To  address  the  violation  of  the  normality  assumption,  different  data 
transformations,  including  log,  logit,  exponential,  and  Box-Cox  were  used.  None  was 
successful,  because  normality  tests  are  sensitive  to  outliers,  and  also  because  we  are 
dealing  with  bounded  values  (0,1)  and  a  majority  of  our  data  is  zero  (no  casualties). 
Nevertheless,  the  histogram  and  the  normal  quantile  plot  of  exponential  transformation 
looked  quite  normal  (Figure  30). 
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Again,  our  process  for  creating  the  regression  model  consisted  of  a  forward  BIC 
stepwise  technique  followed  by  standard  least-squares  regression.  Into  the  model  we 
allowed  main  effects  and  two-way  interactions.  We  examined  the  stepwise-regression 
step  history  to  see  the  knee  in  the  curve  of  R-squared  as  new  terms  were  added  to  the 
model.  We  used  this  information  to  guide  us  in  final  selection  of  a  regression  model.  As 
Figure  31  indicates,  we  reach  a  point  where  adding  statistically  significant  terms  has 
essentially  no  practical  impact  on  the  model  of  fit. 
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Figure  3 1 .  R-Squared  Values  of  the  Exponential  Transformed  Fitted  Model  for  Mean 

Blue  Casualties 


Figure  32  displays  the  actual  by  predicted  plot  and  summary  data  for  the  final 
model.  This  model  consists  of  19  terms,  including  the  intercept,  and  explains  more  than 
90%  of  the  variance  within  the  model.  For  this  model,  the  adjusted  R-squared  is 
0.902532. 
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Figure  33  displays  the  set  of  parameter  estimates  and  prediction  profiler.  The  top 
two  influential  factors  are  number  of  pirates  and  pirate  speed.  Both  estimates  are  positive, 
indicating  that  they  yield  greater  blue  casualties.  On  the  other  hand,  the  third  most 
important  regressor  is  convoy  speed,  which  is  negative.  Thus,  increasing  convoy  speed 
results  in  fewer  blue  casualties. 

We  also  note  that  a  lesser,  but  still  significant,  factor  is  the  number  of  merchant 
ships.  Increasing  the  number  of  merchants  leads  to  an  increase  in  blue  casualties. 


Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

NumPirates 

0.1537716 

0.001546 

99.45 

<.0001  * 

PirateSpeed 

0.0160728 

0.000298 

53.97 

i; : 

<.0001  * 

ConvoySpeed 

-0.02326 

0.000504 

-46.16 

i  iL_ 

<.0001  * 

(ConvoySpeed-1 5.1 765)*(NumPirates-2.5) 

-0.013563 

0.000422 

-32.13 

: ;  n 

<.0001  * 

(NumPirates-2.5)*(PirateSpeed-35) 

0.0061274 

0.000252 

24.31 

j: :  i 

<.0001  * 

NumMerchants 

0.0039304 

0.000247 

15.91 

J  ; 

<.0001  * 

(PirateSpeed-35)*(merchantSensRng-4500) 

-6.91 6e-6 

4.461  e-7 

-15.50 

MM: 

:  :  :  : 

<.0001  * 

(NumMerchants-20)*(NumPirates-2.5) 

0.0030183 

0.000209 

14.43 

]:  : 

<.0001  * 

warshipPos{2-3&1} 

-0.020376 

0.001819 

-11.20 

<.0001  * 

(ConvoySpeed-1 5.1 765)*(PirateSpeed-35) 

-0.000813 

8.226e-5 

-9.88 

<.0001  * 

(warshipPos{2-3&1}+0.33333)*(NumPirates-2.5) 

-0.014871 

0.001527 

-9.74 

<.0001  * 

HeloTakeOffDelay 

0.0010639 

0.000126 

8.45 

<.0001  * 

(ConvoySpeed-1 5.1 765)*(NumMerchants-20) 

0.0005716 

7.239e-5 

7.90 

<.0001  * 

(ConvoySpeed-1 5.1 765)*(merchantSensRng-4500) 

4.3576e-6 

6.883e-7 

6.33 

<.0001  * 

(CommLatency-5. 52381  )*(merchantSensRng-4500) 

-5.231  e-6 

9.451  e-7 

-5.53 

<.0001  * 

merchantSensRng 

-0.000013 

2.51 3e-6 

-5.19 

<.0001  * 

CommLatency 

-0.003008 

0.000606 

-4.97 

<.0001  * 

(HeloTakeOffDelay-36)*(NumPirates-2.5) 

0.0004925 

0.000106 

4.63 

<.0001  * 

Prediction  Profiler 


-0.3333 


36 


4500 


warshipPos{2  15.176  20  HeloTakeOffDel  2.5  5.5238  35  merchantSensR 

3&1  ConvoySpee  NumMerchant  ay  NumPirate  CommLatenc  PirateSpee  n9 


Figure  33.  Parameter  Estimates  and  Prediction  Profiler  for  Final  Fitted  Model 


Figure  34  displays  the  interaction  profiler  generated  by  JMP.  In  this  plot,  the  y- 
axis  represents  the  mean  blue-casualties  value.  Strong  interactions  are  seen  as  highly 
nonparallel  lines  in  interaction  plots.  Here,  the  strongest  interaction  occurs  between  the 
convoy  speed  and  number  of  pirates.  In  this  case,  increasing  the  number  of  pirates  has 
great  effect  on  increasing  blue  casualties  when  the  convoy  speed  is  low.  We  can  also  note 
that  the  warship  can  adequately  handle  the  case  where  there  is  only  one  pirate, 
independent  of  convoy  speed. 
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Finally,  a  significant  interaction  occurs  between  the  number  of  pirates  and  the 
number  of  merchant  ships.  When  the  number  of  pirates  increases,  the  blue  casualties  are 
greater  with  an  increasing  number  of  merchant  ships.  Again,  it  is  important  to  note  that  in 
the  case  of  one  pirate,  the  blue  casualties  are  independent  from  the  number  of  merchants. 
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Figure  34.  Interaction  Profiler  for  Final  Fitted  Model  of  Mean  Blue  Casualties 


B.  CLASSIFICATION  AND  REGRESSION  TREES 

The  use  of  classification  and  regression  (or  partition)  trees  (CART)  is  an  easy  way 
to  quickly  look  at  the  data  and  gather  the  important  relationships  between  factors  and 
responses.  We  call  the  procedure  “classification”  when  the  response  variable  is  discrete 
and  “regression”  when  the  response  variable  is  continuous.  CART  is  a  partitioning  of  the 
data  that  occurs  consecutively  according  to  the  optimal  splitting  value  determined  from 
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all  possible  values  of  each  variable.  The  pictorial  representation  of  the  CART  is  an 
upside-down  tree  with  the  root  at  the  top,  with  branches  and  leaves  below  (Montgomery, 
Peck,  &  Vining,  2006). 


Figure  35  displays  a  recursive  split  of  mean  blue  casualties  over  all  decision  and 
noise  factors.  As  partitioning  of  the  data  proceeds,  the  most  significant  factors  and  split 
values  produce  the  leaves  of  the  tree.  The  splitting  point  for  each  factor  may  suggest  a 
lower  or  upper  limit.  Each  box  includes  the  number  of  data  points  in  the  split,  as  well  as 
the  mean  and  the  standard  deviation  of  the  blue  casualties.  In  this  particular  tree,  we 
chose  to  stop  at  nine  splits,  yielding  a  partition  tree  model  that  explains  almost  85%  of 
observed  variability. 


Figure  35.  Partition  Tree  for  Mean  Blue  Casualties 


The  first  split  is  made  on  the  number  of  pirates.  This  is  the  most  significant  factor 
in  the  scenario.  The  optimal  split  point  is  at  the  value  of  three  pirates  and  any  number 
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smaller  than  that  can  be  expected  to  result  in  significantly  fewer  blue  casualties.  The  next 
split  is  in  the  merchant-sensor  range,  which  is  the  second  most  significant  factor.  A 
merchant-sensor  range  equal  or  greater  to  4,000  m  results  in  fewer  blue  casualties.  The 
third  split  occurs  in  convoy  speed,  where  a  speed  greater  or  equal  to  14  knots  results  in 
fewer  blue  casualties.  The  fourth  split  is  pirate  speed.  A  speed  less  than  39  knots  results 
in  fewer  blue  casualties.  We  finally  note  that  when  the  number  of  pirates  in  the  scenario 
is  greater  or  equal  to  four,  then  a  number  of  merchant  ships  less  or  equal  to  14  provides 
better  protection. 

C.  ROBUST  ANALYSIS 

We  first  defined  our  target  value  x  to  be  equal  to  zero  blue  casualties.  We  then 
created  a  new  column  for  the  loss,  making  use  of  the  quadratic  loss  function:  l  (blueCas) 

=  (blueCas  -  x) 2.  Finally,  we  summarized  over  the  noise  factors,  calculating  the  mean 
loss  and  mean  and  standard  deviation  of  the  response  blueCas.  This  new  data  table 
contained  fifty-one  rows. 

We  first  examine  the  partition  tree  over  the  mean  loss,  displayed  in  Figure  36. 
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Figure  36.  Partition  Tree  for  the  Mean  Loss 


The  first  split  of  the  data  occurs  in  the  convoy  speed,  where  convoy  speed  greater 
or  equal  to  18  knots  yields  better  results  in  the  protection  of  the  merchant  ships.  The 
second  most  significant  factor  is  the  number  of  merchant  ships  in  the  convoy.  Fewer  than 
14  merchant  ships  results  in  a  lower  number  of  blue  casualties  than  if  the  number  of 
merchants  is  greater  than  or  equal  to  14.  The  third  split  is  made  on  warship  position, 
which  is  the  third  most  important  factor  for  the  robust  analysis.  A  position  on  the  flanks 
or  in  front  of  the  convoy  results  in  fewer  blue  casualties;  sailing  behind  the  convoy 
results  in  greater  blue  casualties. 

Analyzing  the  data  for  the  mean  loss,  we  see  that  the  distribution  is  quite  normal 
with  a  mean  of  0.176  and  standard  deviation  of  0.067,  with  maximum  loss  of  0.31,  as 
shown  in  Figure  37. 
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Figure  37.  Distribution  for  the  Mean  Loss 


For  the  robust  analysis,  a  regression  model  was  created.  The  process  for  creating 
the  regression  model  consisted  of  a  forward  BIC  stepwise  technique  followed  by  standard 
least-squares  regression.  Into  the  model  we  allowed  main  effects,  two-way  interactions, 
and  second-order  polynomial  terms.  We  examined  the  stepwise-regression  step  history  in 
order  to  see  the  knee  in  the  curve  of  R-squared  as  new  terms  were  added  to  the  model. 
(Figure  38) 


Figure  38.  R-Square  Values  of  the  Fitted  Model  for  Mean  Loss 
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In  contrast  to  what  we  saw  above  in  the  analysis  of  the  residuals  for  the  whole 
model,  the  analysis  of  the  residuals  here  indicates  that  all  four  assumptions  are  satisfied. 
As  shown  in  Figure  39,  the  residuals  pass  the  normality  test. 


Figure  39.  Distribution  of  the  Residuals  for  the  Mean  Loss 


Figure  40  displays  the  actual  value  by  predicted  plot  and  summary  data  for  the 
final  model.  This  model,  consisting  of  13  terms,  including  the  intercept,  explains  more 
than  98%  of  the  variance  within  the  model.  For  that  model,  the  adjusted  R-squared  is 
0.9845. 
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Figure  40.  Fitted  Model  for  Mean  Loss 


Figure  41  displays  the  set  of  parameter  estimates  and  the  prediction  profiler.  The 
top  two  influential  factors  are  convoy  speed  and  number  of  merchants.  The  convoy  speed 
parameter  is  negative,  indicating  that  it  yields  lower  mean  loss,  while  the  number  of 
merchants  is  positive,  indicating  greater  mean  loss. 
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Figure  41 .  Parameter  Estimates  and  Prediction  Profiler  for  Final  Fitted  Model 
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Figure  42  displays  the  interaction  profiler  generated  by  JMP.  In  this  plot,  the  y- 
axis  represents  the  mean  loss  value.  Here,  the  strongest  interaction  occurs  between 
convoy  speed  and  the  number  of  merchant  ships.  In  this  case,  increasing  the  number  of 
merchants  has  great  effect  on  increasing  the  mean  loss  when  the  convoy  speed  is  high. 
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Figure  42. 


Interaction  Profiler  for  Final  Fitted  Model  of  Mean  Blue  Casualties 


D.  PARALLEL  PLOTS 

Parallel  plots  provide  a  visual  representation  of  the  interaction  between  the 
variables.  In  Figure  43,  2,142  lines  connect  the  response  variable  with  2,142  DPs.  Traces 
associated  with  the  higher-  and  lower  mean  blue  casualties  are  highlighted  with  red  and 
blue  solid  lines,  respectively.  Translating  the  insights  of  the  parallel  plot  in  the  context  of 
the  scenario,  positioning  the  warship  in  front  of  the  convoy  results  in  the  lowest  mean 
blue  casualties,  whereas  placing  it  behind  of  the  convoy  results  in  the  highest  mean  blue 
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casualties.  As  stated  earlier  in  the  regression  analysis,  however,  the  position  of  the  ship  is 
not  a  significant  factor  of  the  model.  This  result  is  reinforced  with  the  parallel  plot,  since 
the  position  of  the  warship  in  front  of  the  convoy  results  in  some  of  the  highest  mean  blue 
casualties  as  well.  Moreover,  the  position  of  the  warship  behind  the  convoy  yields  some 
of  the  lowest  mean  blue  casualties. 


An  important  takeaway  from  the  parallel  plot,  which  reinforces  the  results  from 
the  regression  analysis,  is  that  the  number  of  pirates  is  a  significant  factor.  The  higher  the 
number  of  pirates,  the  higher  the  mean  blue  casualties  is. 


Finally,  pirate  speed  is  another  significant  factor  of  the  model,  since  high  pirate 
speed  results  in  high  blue  casualties  and  low  pirate  speed  results  in  low  blue  casualties. 


Figure  43.  Parallel  Plot  for  Mean  Blue  Casualties 


For  the  robust  analysis,  51  lines  representing  the  51  design  points  are  shown  in 
Figure  44.  The  plot  indicates  that  high  convoy  speed  yields  low  mean  loss,  while  low 
convoy  speed  results  in  high  mean  loss. 


54 


Figure  44.  Parallel  Plot  for  the  Mean  Loss 
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VI.  CONCLUSIONS 


A.  ANALYSIS  SUMMARY 

This  research  set  out  to  determine  how  well  an  escorting  warship  can  protect  a 
convoy  of  merchant  ships  while  handling  pirate  attacks.  In  addition,  we  tried  to  answer 
how  many  merchant  ships  a  warship  can  adequately  protect  and  what  formation  (speed, 
distance,  position  of  the  warship)  the  convoy  should  follow  to  safely  travel  through  the 
IRTC. 

This  study  mainly  uses  three  analysis  techniques  to  look  at  the  mean  merchant 
(blue)  casualties:  multiple-regression  analysis,  robust  analysis,  and  classification  and 
regression  trees  based  on  over  300,000  simulated  convoy  missions. 

The  three  analyses  complement  each  other.  Each  analysis  identifies  similar  factors 
of  greatest  importance  and  key  interactions  and  provides  similar  insights.  Moreover,  the 
regression  analysis  yields  a  formula  for  predicting  mean  merchant  casualties  and  insight 
into  the  significant  factors  of  the  scenario.  The  three  most  influential  factors  are  the 
number  of  pirates,  pirate  speed,  and  convoy  speed.  Additionally,  the  regression  analysis 
reveals  some  interesting  interactions  between  the  important  factors.  Increasing  the 
number  of  pirates  has  great  effect  on  increasing  blue  casualties  when  the  convoy  speed  is 
low.  Another  significant  interaction  occurs  between  the  number  of  pirates  and  the  number 
of  merchant  ships.  When  the  number  of  pirates  increases,  blue  casualties  are  greater  for 
increasing  numbers  of  merchant  ships.  Finally,  the  regression  notes  that  the  warship  can 
adequately  handle  the  case  where  there  is  only  one  pirate,  independent  of  convoy  speed, 
and  that  increasing  the  number  of  merchants  leads  to  an  increase  in  blue  casualties. 

For  the  robust  analysis,  a  regression  model  was  also  created,  providing  a 
prediction  formula  based  on  the  decision  factors  of  the  scenario  only.  The  two  most 
influential  factors  are  convoy  speed  and  number  of  merchant  ships.  The  interaction  that 
occurs  between  the  convoy  speed  and  the  number  of  merchant  ships  indicates  that 
increasing  the  number  of  merchants  has  great  effect  on  increasing  the  mean  loss  when  the 
convoy  speed  is  high. 
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Two  separate  classification  and  regression  trees  were  conducted:  one  for  the 
robust  analysis  over  decision  factors  and  one  over  the  whole  number  of  scenario  factors. 
The  first  CART  indicates  that  a  convoy  speed  greater  than  or  equal  to  18  knots  and  a 
number  of  merchant  ships  in  the  convoy  less  than  14  yields  better  results  in  the  protection 
of  the  convoy.  Also,  positioning  the  warship  on  the  flanks  or  in  front  of  the  convoy 
results  in  fewer  merchant  casualties,  whereas  a  position  behind  the  convoy  results  in 
greater  merchant  casualties.  The  second  CART  indicates  that  the  most  significant  factor 
of  the  scenario  is  the  number  of  pirates.  Three  pirates  or  fewer  can  be  expected  to  result 
in  significantly  fewer  merchant  casualties.  Two  other  important  takeaways  from  the  later 
CART  are  the  merchant  sensor  range  and  convoy  speed,  where  values  equal  to  or  greater 
than  4,000  m  and  14  knots,  respectively,  result  in  fewer  merchant  casualties.  Finally,  a 
pirate  speed  of  less  than  39  knots  results  in  fewer  merchant  casualties  than  a  pirate  speed 
greater  or  equal  to  39  knots. 

B.  FOLLOW-ON  WORK 

This  thesis  provides  many  opportunities  for  follow-on  research: 

•  The  scenario  may  be  expanded  using  more  than  one  escort  warship  or 
Unmanned,  Aerial  Vehicles  (UAVs)  to  patrol  over  the  convoy  for  the 
whole  or  part  of  the  trip. 

•  The  scenario  may  also  include  other  MOEs,  such  as  the  time  it  takes  for 
the  warship  to  interdict  the  pirates. 

•  Finally,  the  scenario  may  be  expanded  to  a  higher  level  of  detail.  It  may 
include  the  time  it  takes  the  warship  or  helicopter  to  stop  and  arrest  the 
pirates  or  the  actual  self-protection  measures  the  merchants  may  take  to 
deter  or  delay  the  attack,  like  razor-wire  barriers  or  water,  spray,  and  foam 
monitors. 
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